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INTRCDUCT :cr. 



A. EACKC-RCUND 

STAPPEP.2 (S/stem for Tactical Assessment cf 
Pul ti source Messages, Ever. Rader) is an experimental 
computer program created as part cf an investigation 
into new methods of correlating information available in 
a naval environment in response to toe tactical situation 
assessment (TSA) problem. It was written by the Tactical 
Command and Control Division of the Naval Ocean Systems 
Center, San Diego, California. STAMMERS has served as a 
testbed for explorations of applications of rule-based 
artificial intelligence inference systems, witn a general 
and flexible approach to the range of acceptable inputs 
tc the system. 

In practical application, information in a oasic 
format is received from a variety of sensors. This 
information is processed within a network of 
interconnected rules or conaitional statements which 
may be satisfied by this information, even 
responding to information in cases involving varying 
degrees cf uncertainty. STAMPERS is capable cf 
performing deductions based or. this received i nf ormat ion 
tc speed the decision process in a rapidly changing 
environment . 
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Before we discuss the specifics of STAKf^IR2 and a 
nethoi of testing the utility of this decision aid, 
background information will be presented to set the stage 
for the why, where, and how such a system was developed. 



3. OBJECTIVES 

There ere three principal objectives for this thesis: 



1. Identify the characteristics of an artificial 

intelligence program, with an overview of the 

applications to a specific military situation. 

2. Proncse various levels of evaluation methodologies 
for* STAKPSH2 . 

3. Present the results of an experiment conducted 

with STAMI*IR2 using one of the methodologies 

presented . 
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II. ARTIFICIAL INTZILIGSNCE 



A. DEFINITION 

Artificial intelligence (aI) is defined as "the study 
of ideas which enable ccrnputers tc dc the things that make 
people seen intelligent" r Ref. 1] end "the science of making 
machines do things that would require intelligence if dene 
by men" [Ref. 2]. 

The intricate processes from which these "things" 
have evolved, such as learning, growth, or maturation 
are bypassed in the artificial intelligence program to 
create a facsimile of hum, an thought and, subsequently, 
decision-making. But the bypassing is, in reality, a very 
careful compilation of the elements which are, or seem to 
be, parts of the human theugnt patterns and strategies 
leading to decisions. That is, human thought is analyzed 
and dissected into discrete elements which may be 
recombined artificially to mimic human intelligence. 

Alternatively, differing patterns or methods are 

explored to uncover new routes which could, in fact, lead to 
sirilar results. 

B. DISCUSSION 

To imagine the thought process as a branched tree, 
with decisions made at each juncture, cr node, 
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is a simplification 


which does not 


reflect 


the complexity 


cf tne problem of 


describing 


human 


thought . The 


actual representation 


must 


consider 


many 


interconnected 


paths, with multidire 


ctional 


flow 


through 


those paths. 


A control mechanism 


mus t 


also 


be considered which 



carries a thought through to conclusion and minimises 
retraced cr ncn-prcduct ive routes . 

A goal of artificial intelligence inquiry is to 
duplicate this complex structure in a corrputer program. A 
difficulty lies in the multi-dimensional nature cf the 
thought process which requires equally complex 
investigation. By examining the problem from the viewpoint 
of discrete elements, it ray be possible to find the 
combination which accurately reflects a pattern which net 
only duplicates what happens in the case of a specific 
problem, but also a general case which ray be applicable to 
ether pr colens . 

If STAMMER2 can duplicate the reasoning process of a 
decision maker, with a data base and memory consistent with 
the situation presented, then it is possible to use this 
artificial intelligence program as an "intelligent 
assistant" in the tactical situation assessment problem 
which will be discussed in the next section. 

The computer is the "ideal experimental animal." 
Infinitely patient and easy to care for, the computer 
requires structured instruction, composed of discrete 
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elements, which it combines as programmed . The combinaticr. 
end recombination, done at speeds sufficient to expose 
unproductive ratns rapidly, can provide a practical test of 
the precision with whicn the experimenter defines his 
thought model [Ref. 3]. Additions or deletions which 
the experimenter may hypothesize as crucial (or useless) to 
the final result of his pattern may be evaluated with 
relative ease. Flaws or conceptual mistakes which may 
exist are likely tc be detected in the computer because 
the computer generally will not accept ambiguity of 
instruction and may simply cease execution of the 
program. However, a program designed tc answer a certain 
sort cf question does not gua rantee a correct answer 
[Ref. 4] . 

The applicability cf computers and programming is net 
the emphasis cf artificial intelligence. However, the 
unambiguous nature cf the program is mentioned to reinforce 
the ” transparency" cf the actual machine and focus on the 
precision cf understanding human thought. 

Using artificial intelligence techniques, research has 
been done to explore vision, problem solving, language, and 
neurosis [Ref. 5], Some of these models have been 
successful in predicting behavior under specified conditions 
while others have been at least of metaphorical value in 
helping t c understand the possible mechanism cf 
intelligence. By representing thought and knowledge, in 
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general, in a useful and flexible manner it is also possible 
tc arrange an interaction between a human and the machine in 

a task; oriented environment. 

Knowledge is defined as a collection of facts, the 
state cf knowing, or all that has been perceived by the 
human mind [Ref. 6]. Knowledge ray be in human or machine 
memcry or in a data base. It is, however, also in the 
actual procedures that operate on or by reference to the 
data. Tc store disjointed facts in computer memcry or in 
a data base from which the labelled information units may 
be retrieved is not the goal of artificial 
intelligence. The identification cf the desired fact (or a 



location 


code 


by which 


that fact 


may 


be retrieved) is 


a 


problem 


of 


database 


management . 


Tc 


discern which fact 


is 


re » ui red 


is , 


however, 


the end 


of a 


series of steps . 


In 


t h e problem 


solving 


case, the 


solution process is 


the 


strategy 


of 


gaining 


inf orma tion 


after ’’ a forward 


or 



backward looking reasoning method" [Ref 7] . 

Analogies to other human processes exist. Tor example, 
radar provides target information in a nanner analogous 
tc the eye. Data, such as target presence, range, and 
bearing may be useful to the operator as well as in a form 
which can be compared tc characteristics stored in computer 
memory . 

The specific goal of STAPMER2 is to practically combine 
this sensory data with the problem solving process of 
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data receipt, cor.parison and retrieval using 



the 



knowledge 



stored 



ir. the artificial intelligence program. 
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III. TACTICAL SITUATION ASS iSSf-T^ T AN I STAWZRZ 



A. TACTICAL SITUATION ASSISSMINT 



In the naval context, the atility cf a commander tc 
accurately assess his ml litany situation requires decisions 
about his environment. While all decisions need not be rade 
cn an immediate basis, certain cf then, called tactical, 
determine his immediate course of action. In ar. extreme 
example, the decision tc launcr. a defensive weapon is based 
on c judgment that such an action is required based cn the 
commander's evaluation. A statement of the tactical 
situation assessment (ISA) prcbleT includes consideration 
cf the inforration available tc the decision maker. 

In order tc assess a military problem, the evaluator 
draws upon experience and training, after having considered 
the situation presented tc aim. The assessment cf the 
tactical situation is, however, highly dependent upon 
factors associated with the decision maker himself. 

The experience upon which a military decision maker 
draws is unique tc him alcne, although there is 
certainly a similarity of career patterns end situations 
common to many individuals. Unique to that individual is 
the cuality of his recollect icn , his emotional cr mental 
state, the perspective of the individual, and the length 
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c f time elapsed since tries e events f r cm which the 
experience was gained. 

The actual decision maker in the operations center 
the Combat Information Center or CIC) of a Navy ship is 
the Tactical Action Officer (TAO). During the lericd of 
the Vietnam conflict, tne rcle cf the Captain cf a ship as 
final fighting authority was altered somewhat from the 
traditional rcle. The Tactical Acticn Officer concept was 
proposed, implemented in Navy Regulations, and exercised in 
combat [Ref. 6]. The concept basicly states that a trusted 
subordinate, trained end drilled in defensive doctrine 
and procedures, acts as weapons release authority in the 
temperary absence cf the Captain. 

The specialized formal training for TAOs, having as 
prerequisites proven maturity and operational 

experience, lasts approximately six weeks. The student is 
exposed to information concerning U.S. weapons systems 
and those of potential enemies. After this is 
committed to memory, "standard" tactical doctrine is 
exercised in a series cf increasingly complex scenarios. 
The scenarios are presented in sequences allowing 
Questions tc be answered based cn information normally 
available tc the TAC (from the training, experience, 
tactical publ icat ions , sensors , and intelligence sources). 

The TA.0 becomes the focus of the Combat Inf orma t ion 
Center. All sersor reports are displayed to him, all 
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messages from external sources are giver, tc him directlv or 
in summarized form. rie can stand alcr.e, cn nis cvr. 
ship, or be electronically linked by radio tc TAOs on 
ether ships, each going through similar procedures and 
actions. This defines a sphere of knowledge in which the 
I AO operates, ana can generally be delineated by tne range 
of his sensors. A correlation function is required when 
the TAO receives 'external" reports from sensors cr 
platforms ether than his cwn — in effect from a sphere of 
larger radius. 

Assumptions involved in reducing many TAO activities 
to computer dorr include: a) the basic, repetitive mass 
of information can be reducea to a computerized database; 
b } standardized procedures for data retrieval and 
correlation can be sirilarly reduced; c)the 
sensor/intelligence information car. be presented in a 
standard format; and d) the thought process by which the 
TAO functions may be artificially reproduced. 

r. SI AMMZF.2 

STAMMIR2 is a revised version cf STALER, a System for 
Tactical Assessment of Multisource Messages, Ever. Rader. It 
was created as cart cf an investigation cf correlation 
methodologies, and served as a testbed for explorations of 
applications of rule-based inference systems to the tactical 
situation assessment -TSA) problem. STAMMER concentrated cr. 
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the specific task of merchant ship disc riri nation frcr all 
contacts reported by radar and external messages ]?.ef . . 

S 7 ml P.2 is an organizer of information. It 
collects information cy receiving • r essa^es and sensor 
reports (radar, electronic support measures, end sonar), 
and organizes this raw data intc graphic displays and 
textual c cr.rr.enta r y to aid in tactical situation 
assessment. Thrcug.n the use of specified rules, the 
system cc~bines tnis information to drav conclusions 
about the situation in the vicinity of the home ship. Tncse 
combinations are reflected in both the STAkKI?.2 display and 
cor renter/. The system is available for the examination of 
raw data as well as information about why and hcv the 
conclusions were reached. 

STAi v i v Z?.2 deals with information cn a real-time basis. 
As a message is received from a sensor or information 
source in SkAhkIP.2- reada ole format, it becomes part of 
the data base cn which the rules operate. This process is 
complicated by the fact that the information is suspect, and 
its arrival may net be in chronological order. 

Although S7 aN>'I? 2 was developed in response tc 
Navy- sponsored investigation of the TSA problem, the U.S. 
Army nas conducted similar investigations in the 
field of military intelligence analysis [Ref. 10]. 
rases, upon a detailed stuay of the analysts' role, 
methods, end thought processes in intelligence production, 
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it may be possible to streamline intelligence analysis. 



In essence, 


tnis process 


used by 


the analyst is 


the 


rethodolc gy 


from which 


the AI 


program might 


be 



developed after a descriptive model is organized and 
exercised . 

The manner in which STAMPERS functions ray best be 
described by analogy to the medical diagnosis system called 
t v YCIh [Ref. 11]. f-’YCIN is a problem solving system by 
wuiich a physician, responding to emeries from the program, 
receives assistance in identifying and treating blood 
bacterial infections. The basis for the expertise 
resident in f-' Y G IN is the knowledge obtained from 
interviewing physicians and experts and stored in over Z? 2 
"productions". Productions are memory structures on which 
the program operates and are in the following format. 

If the infection type is primary-bacteremia , 

the suspectea entry ‘point is the gastrointestinal tract, 
and the site of the culture is one of the sterile sites, 

then tnere is evidence that the organism is bacteroides. 

Conceptually , inference rules ere very simple. Every 
rule attempts to retrieve information from memory and, if it 
succeeds, constructs a new assertion in memory. If all the 
conditions are matched, then the rules are said to "fire". 
That is, ail the actions which are defined in the rule are 



then carried out. These actions may be modifications to tne 
data base, assigning of confidences, and printing of 
formatted descriptions. 

In a sirple rule interpreter, the system maintains a 
list cf rules, each witn conditions which pertain to a 
central data base, whicn is constantly being updated. At 
regular intervals the system attempts to fire each rule; 
that is, it checks to see if the rule conditions are 
currently satisfied in the data base. If ere of many 
conditions is not satisfied, a rule ray fail to fire and, in 
this case, any partial work dene in satisfying the rule is 
lest. In addition, each rule is retried at the time 
interval without regard to the nature cf the intervening 
changes to the data oase, duplicating results where no 
changes to the data base nave occurred. 

Because medical diagnesis, even by experts, involves 
varying degrees of uncertainty, I^YCIN is written with a 
so-called certainty factor fer each ccnclusicn. The 
physicians upon whose information the productions were 
established were also asked tc provide "strength 
values", or probabilities cf accuracy and confidence, 
for their assertions. This introduces the ability to 
deal with uncertain evidence into the artificial 
intelligence system. 

The productions used to reach a conclusion are part cf 
the conclusion set. KYCIN answers questions about how end 
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v.h y a fact v.as established or used. The production, is 
recalled n are cr number; tc shew an enumeration cf the 
facts presented in the premise set. General facts can he 
o.uesticned wit heut resorting tc tne entire diagnosis because 
each production 'stands alone", to be actively examined or 
cuest ionec . In this case the steus taken to arrive at that 



particular fact can else be traced. This tracing 



ma. 



act 



as a training aid for the user in addition to presenting 
alternatives which can be explored. 

The critical aspects of STAMMERS 's design may be divided 
into four parts. These are memcry (the data base), rule 
interpretation, explanation, and graphics. 

The rules of STAMPERS consist of conditions, actions and 
confidences, with conditions and actions in the sere 
structure as assertions with allowances made for variables 
which are "tcund" tc a rule by a binding function, in LIS?, 
the language in which STAMMER? is written [Ref. I2j . These 
variables which ere applied tc the rule conditions will be 
evaluated to see whether tne condition succeeds or fails. 
If it fails, a "suspension" is created which corresponds tc 
the remainder of the rule. As more information is added to 
the data base, these suspensions which can use the new date 
are revived and continue as before (eitner completing or 
suspending again). A suspension contains net only the 
remaining part of the rule, but also the bindings 



established by already satisfied conditions, if any. 



Even 



when c condition succeeds, there may be other ways for it tc 
be satisfied, so a suspension is left behind. 

The individual rules are expressed in the f cl lew ins; 
fern . 



RULi-N A ME 
(CONDITIONS 
( ( <ccndition 1> ) 
i <ccnditicn 2> ; 

• 

(. <co edition n> }) 

ACTIONS 

1 ( <actior. 1 > ) 

( <action 2> ) 

( <actior. m> ) 

CON? <a number between -1 and +1' 

PR IN FORM 

"This is a sentence describing the rule.") 

STAMMER2 deals witn information cn a real-time 
basis. However, this process is complicated by the fact 
that the arrival of reports may net be in chronological 
order, with later information superseding or negating 
earlier reports. A ’data stream" is used to bind 

information as it is received to t.ie rule condition which 
it satisfies. 

A data stream ray be defined as a sequence cf 

values, existing over time in a computation. If a program is 
executed in a conventional language, then the history cf 
successive values of a variable forms a stream. Thus, in 
contrast tc static data structures such as lists and arrays, 



where ell the elements exist e t cne tire, s t rears a r e 
dynamic aata structures, ana are addressable objects in 
1ISF. Only the re* data received need be compared tc the 
rules. New matches are added to the end of the stream, h 
temporary "freezing” of the action specified by an assertion 
occurs until after the new date is read onto the appropriate 
streams. The stream, then, allows the rapid review of stored 
information w i t nout the penalty c f complete review of 
potentially irrelevant date. 

I r. STANKZR2, confidence is provided in the rules by the 
creator of the rule. It is calculated dynamically on 
request at the time when it is displayed to the operator 
following a rule match and is net stored. Each assertion 
will have as its confidence the con bination of the 
confidences of the rules and assertions which offer evidence 
for it. A confidence calculation is highly dependent upon 
the connective through whicn the rules and assertions are 
combined. An AND connective will, fer example, display the 
confidence of the snallest value of all the rules combined. 
That is, the conclusion may never be stronger tnan the 
weakest piece of supporting evidence. 

During execution of STAi y KZR2 , the user will see 
the following cycle repeated as long as messages ar.d reports 
are received into the message input files 

1. A message report is received, the user is informed, 
and the critical information in the message is printed 
for reference. 
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2. A display , showing the area situation with the new 
information. is uraua. The user m a/ manipulate this 
image . 

3. The system makes some commentary on the conclusions it 
can reach, on the basis of the new information. 

4. If any conclusions were reached, the user is driven the 
opportunity to ouery the system about the contents of 
its data base. 

The message text is in the basic format of the received 
message. A contact name (if available), is displayed, 
followed by location end time cf message. The user may pause 
tc examine this message or select the graphic display mode. 

Graphical support for STAMMER 2 is provided by 
TSPLA, a software package developed at the Naval Ocean 
System Center specifically fcr tactical situation 

assessment [Ref. 13]. The DSPLA system is a collection of 
PORT R A K subroutines teat allow storage, retrieval, and 
cispla/ of ship ana aircraft tracks. This display, with the 
capability tc shew ~a ps with latitude and longitude, -"ay 
be manipulated by the user tc vary the scale as desired. As 
seen as a rressage is received which contains a 

demonstrable assertion (such as a contact at a specific 
location) the display will appear on the display screen 
•if a display is available and selected in the 

initialization process). Examples of basic function keys ere 
are listed below. 

K magnify about the cursor by a factor of 2 

?. reduce about the cursor by a factor cf 2 



V center the view a t cut tne cursor pc s i t i cr 
; set type size to smallest on the TEKTRONIX -=214: 

C return t c command .node 

More detailed functions exist, with the full listing in f.CSC 
Technical Document 252. The user rray return to the 
command level to begin the explanation end query procedure. 

The explanation s/ster provides two primary 
capabilities: retrieval of memory and inference tracing, 
although there would appear to be little difficulty in 
retrieving memcry contents and tracing derivations, the 
display of this information in a human readable format is a 
major consideration. To make the user interface as natural 
as possible, the explanation system provides a query 
language that is ’ English-like" . This language is an 
extremely limited version of English, which includes only 
certain types of questions and methods of phrasing those 
questions. However, the language was designed sc that, 
while limited, it is sufficient to cover the user's needs 
without making its shortcomings apparent. The LIS? function 
iiSEUSEP, which features recognition and prompting, is 
combined witn a ’ prettypri nter" (to add words like ’is’ and 
"of” to memory contents) to simplify the user interaction. 

A sumpary of the queries which the STAMMERS user can 
ask, with general examples, follows. Hollowing the 
initial query word, only tncse inputs shown will be allowed 
by the A.SKUSER function. The connective words (in lower 
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case) will be provided by the "prettyprinter" [Ref. 14]. 
The words in parentheses may be inserted by the user as 
desi red . 

WHAT Format: 'WHAT is l THE AN A) < RELATION/ (OF) 

< I TEbv 

Example: WHAT is the COURSE of SIGHTING3 
This every provides one of several Question 
formats for asking about entries in the data 
base. 

S Format: IS (THE) OTEtO {k AN THE ) <RELATI0N> 

(OF) <ITEM> 

Example : I S RADAR the SOURCE of SIGHTING3 
This question form allows the user to ask 
about specific entries in the data base. 

TELL me about Format: TELL me about <ANYTHING> 

Example: TELL me about SIGHTINGS 
This is the most flexible query, and allows 
the user to ask rules, categories, 
relations, or specific facts. 



VHZRi 



Format: WHERE is <03 JaCT> or WHERE was 

<CEJZCT> at <TirZ> 

Example: WHERE is CONTACT? 

With this command the user can ask about the 
position of a platform, merchant lane, or 
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storm at the present tine, or in the future 
or pest. 

Format : WHY is <ASSSRT ION> 

Example: WHY is A30245 

With this command the user can find out the 
primary or immediate reasons that STAMMER2 
used to conclude any assertion. All the 
rules involved in the decision will be 
displayed . 

Format: KCW does rule <RULE> apply to 

<ASS2RTICN> 

Example: HOW does rule ID-LANE apply to AZZ4 
This query allows the user to find cut what 
assertions or facts were involved in 
permitting the rule to help conclude the 
given assertion. 

Jcrmat: WHOSE <RELAT ION> is <ITEM> 

Example: WHOSE TYPE is MERCHANT 

This query acts as a partial inverse to the 

WHAT query . 

Format: WHO is (THE AN A.' <RELATI CN> (OF ) 

<ITEK> 

Example: WHO is INSIDE LAN El 
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This provides another Torn"' for sue ry i ns the 
data base. 
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IV . 

This section presents a discussion cf experimental 
methodology which may be applied in the evaluation of 
STAMMIE2. Following an overview of general issues, and a 
discussion cf general levels cf experimentation, three 
specific stages of experiment are presented. From these, one 
has been chosen to be carried cut as a sample experiment. 

The design of an experiment is based on the question to 
be addressed. An experiment to compare systems with 
different speeds of message receipt, for example, will 
include a message generator, an interval for testing, and 
the means to measure differences in total messages 
received. The criteria for "better" may be the higher number 
cf messages received in a certain time or the shorter time 
needed to receive a. certain number of messages. These 
objective measures cf performance are established prior 
to the experiment, with sufficient flexibility to present 
the data to satisfy the question. 

In more complex evaluation, the questions to be 
answered may be difficult to define. If the goal of the 
experiment is to find the number of messages received and 
understood during a definite time interval, the criteria for 
understanding must also become a part of the experimental 
procedure. Interpretation of the results cf such an 
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experiment then becomes dependent cn an "acceptable " level 
of understanding, an acceptable number of messages, or a 
r oml ina t ion of the two. This ray then be ar. area where the 
objective measures five way to less precise but equally 
useful subjective evaluation. 

In order tc test the suitability of an 
artificial intelligence system, measures of performance 
may be difficult t c establish. The components cf the 
program may ce individually examined (the structure, 
execution, results), or the overall ’improvement" in a 
parameter of the function which the program is designed to 
assist may be tested. However, to show that the system 
would respond tc a question cr situation as euickly as 
a human decision rarer, given that the situation were 
ccnstraired tc vnat the program and the human knew and the 
information flow was at "normal" speed, may prove 
nothing mere than that computers are faster than humans. 
Considering the difficulty in establishing 
meaningful measures of effectiveness, subjective evaluation 
cf the capability and the utility cf STAb'MEP.2. by tactically 
oriented individuals, may show the greatest promise as an 
evaluation technique. 

Experimentation may be performed ir. stages ranging from 
simple static debug/validation experiments, which 
concentrate on tne capabilities of the technology arc 
require few resources, to more extensive technical 
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evaluations cf STAt- M SR2 with a d y^a^ ically changing lata 
base, in a controlled environment. This could include 
dynamic operational evaluations in simulated ccrrmana and 
control environments pitting opposing forces against each 
other under various scenarios using one of the warfare 
environment sirulator (YES) war game programs [P.ef. 15]. 

following an overview of three possible levels of 
testing, more detailed procedures for these levels will he 
presented . 

The basic level of experiment consists of imputing into 
a data base various static threat scenarios (a snapshot 
view of trie situation} concerning the status of 
’Slue" and ' Cran^e’ task forces. The test objective 
is tc evaluate the capabilities cf STAPLERS tc read the data 
base, signal the existence of the threats, and form 
opinions based cn specific rules selected ty the test 
conductor) concerning other information in the form of 
sensor and intelligence reports. 

The next level of experiment is conducted in a 
more realistic command and control environment with a 



dynamically cnanging data base. A detailed scenario is 

(or its variants), presenting a wide 
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via WES 
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.the limits 


cf 


s ity 


is required 



the length of t i m e fror the instant tne retort is received 
until ST Af^ERA signals a conclusion. 

In the r:ost complex level of testing, W IS is again used 
wit n Orange and rlue task forces. These forces are 
composed as closely as possible, within the capabilities 
of available test resources, with equipment (sensors, 
weapons, etc) end platforms that represent current military 
inventories. Experimental emphasis is shifted fro-" a 
technical evaluation to an operational evaluation of the 
value cf STAMMERS. to tne decision maker. The objective 
is to determine if STAMMERS. improves the commander's ability 
to maue rapid and accurate decisions. Operationally 
realistic scenarios are the desired norm. Each scenario 
should be replicated with different players with s c-e 
trials having the Plue forces operating with STAMMSR2 and 
seme without. Operational measures cf effectiveness, 
which depend or. the scene ri os under play , are selected to 
assess the operational utility of 3TAMMZR2. In addition, the 
players and the umpire team are subjected to pcst-exercise 
interviews and questionnaires to get subjective evaluations 
cf the worth of STAMMEP2. 

While there are no definitive boundaries between the 
levels cf the formal experiments, the rar.?e cf conditions 
must be designed for scenarios that are both realistic and 
informative. The actual analysis at each level of experiment 
Tust take into account the limitations cf the scenario 
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generator r ?.ef. 16]. The three proposed levels of experiment 
* ill new be presented in detail. 

A. ST AGE -ON I EXPERIMENTS 

1 . Objective 

Stage-one experiments evaluate the capability c: 
STAMMEE2 to identify, in a static environment, the tactical 
situation that confronts an afloat task force and to 
describe the situation semantically and graphically. This 
tyre of experiment establishes the type of threats that 
STAMMIR2 can evaluate end assesses the timeliness of the 
warnings. The experiments may also serve to provide feedback 
through which STAMMER2 may be refined. 

2 . Resources Required 

a. Technical 

The STAKMZF.2 program and associated computer 
card ware are required for this experiment. At present, 
STA.f-’MIR2 is resident on the T0PS22 system at NOSC, Sen 
Tieec, California. 

b. E i splay 

An alphanumeric display system will be required 
for players and the umpire team. In addition, a graphical 
display of the situation, as is normally generated by 
V/ES must be available to compare it with that presented by 
STAMME2 . 
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c . Sc ena r i c 

The tactical scenario is presented as a series 
cf operationally realistic ' aata plates’ constructed in 
advance of the experi~ent. Each data plate is a data 
snapshot describing a tactical situation at a given tire. 
The plate snculd include various information iccurse, speed, 
ser.scr reports, intelligence) available to a Tactical Action 
Officer to describe tne situation fully . 
d . Personnel 

The personnel required for the experiment 
include observers of the output (threat assessors), a 
scenario controller, and a test director. 



e. Support Program 

In addition tc scenario generating requirements. 
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2 . General Context 

a. Concept and Need 

STAiYMLF.2 is being developed to interact with a 
graphics display system to support the comrander's decision 
processes. In order tc assess the value cf the technology 
we must determine its capabilities and shortcomings. For 
STA^ V .IR2 tc be useful tc a decision maker it must be able tc 



evaluate threats by a comparison and collation of a 1 1 -sc u rce 
reports and. cispla y the situations in a tirely manner. 

b. General Situation and Scenarios 

A suitable number of data plates should be 
developed to vary the tactical situation significantly in 
both the complexity of the situations and the nature and 
number of the platforms involved. A data plate should be 
selected at random, its number recorded and the 
appropriately stored data input to the file which STACIES 
will read. An observer, unaware of the contents of the date 
plate, ncnitors the contents of the display terminal, 
evaluates the information received, and describes the 
situations as he sees them,. 

4 . lvalue tier. 

a . Data collection 

The following data should be collected for 
each trial: the data plate identification number, the 

situation assessment of tne evaluator, end the 
situation assessment given by STANMSR2. 

b. Analysis 



The primary areas of concern for 
experiments are: a comparison of the £TA^^ZR2 

the human assessments of the threats, and a 
of the times required for STAMUEP2 
different types of situations. 



the stage-one 
output with 
de termi net ion 
to process 
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Post-test analysis will examine tr.e data sheet 
tc ascertain if STAhTERZ accurately evaluated or described 
and displayed all phase-one situations. This is compared 
with the evaluators' ability tc do the same based or the 
same da ta . 

c. Anticipated Results 

It is anticipated that STAhNIEE will signal the 
existence cf tnreats with very little time delay ar.d that 
the situation end display will be presented accurately, 'or 
those cases where complex multiple situations are present 
simultaneously , the tests may indicate an overcrowding of 
the display and a lack cf time to exercise the logic trace 
capability cf STAMfhERE. 

c . Corrr-ents and Srecial Instructions 

Each experimental subject will gc through a short 
ST Ah h'IR2 indoctrination session during which ne is given a 
summary cf tne theory and implementation cf STAhhER2, a 
’threat" briefing to explain the scenario, and ar. overview 
cf what is expected cf him. Care should be taken tc include 
in the data plates as many situations as possible that 
STAhhZR2 has been designed to handle in realistic 
situations. It is recommended that several different 
operators be subjected to the complete set of plates ar.d be 
asked tc subjectively evaluate the displayed information tc 
provide information which may be useful in the design of 
follow-on display experiments. 
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The stage-cne experimer.tat icn discussed accve will 
need to be carried out in the course of a full end careful 
evaluation of STAMM ZP.2 as a tool in tactical assessment. Tor 
the purpose of this thesis, however, only the stage-twc 
experiment (as discussed below) was actually performed. 

2. ST AGS -TWO EXPERIMENTS 

1 . Objective 

The objectives cf stage-twc experiments are: tc 

assess the technical capability of STaMMEPS actions on a 
dynamically changing data base generated by creating 
threat situations through the exercise of WES (or its 
variants); tc determine tne ’limits' of the capability 
of STAMMERS ; and to obtain subjective opinions about 
STAMMZR2 . 

2 . Resources Ree mired 
a . Techni cal 

The STAMMERS program and associated computer 
hardware ere required for this experiment. The Warfare 
Environment Simulator (WES) program, is available on the 
TZNEX system at NOSC. 
b. Display 

An alphanumeric display system is required r'cr 
players and the umpire team. The GENISCC system, a color 
graphic display, may be used tc present the situation 
which is generated cy WES . 
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c . Scenario 

An operationally realistic scenario is 
generated to test STAi v .N!ER2 in a tactical situation at a 
giver, tine. The scenario should include various 
icforration (course, speed, senscr reports, intelligence) 
available to a Tactical Action Officer to describe the 
situation as it is normally presented by a'ES , via 

visual display and status boards. The evaluator will be 
allowed to interact with units under his 'control" by 
ordering course and speed changes, and enabling various 
sensors available to him. The test director nay insert 
various elements of intelligence as desired, 
d . Personnel 

The personnel required for the experiment 
include observers of the output (threat assessors), a 
scenario controller, and a test director, 
e. Support Program 

In addition to scenario generating reaui rener ts , 
software to measure and record the tires at which the 
messages are output and the tines designated by the 
observers is desirable. This program, based on the 

t 

ARPANET TYPESCRIPT feature, records all inputs and 
outputs to selected terminals. 
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f. evaluation quest ior.naire 

Fo llow iag a stud/ cf the measures cf operational 
effectiveness which may he achieved in this interactive 
experiment, an evaluation questionnaire should he 
constructed to establish the level to which these measures 
nave been satisfied. 

3 . General Context 

a. Concept and Need 

A complete technological assessment of the 
capability of STANT 3R2 includes an evaluation in a realistic 
environment with a dynamically changing data base. T he 
experiment must subject STANTZR2 to a wide of situations 
under varying levels of sensor activity in an effort to 

exercise all capabilities of STANNZR2 and try to probe the 
"limits” of those capabili ties . These limits could be the 
size of data base, number of reports, or frequency cf 
reports . 

s b. General Situation and Scenarios 

'aES scenarios will be generated to present a 

variety of sensor reports and levels of activity by 

platforms. An evaluator familiar with tactical war gaming 
will observe the play cf the game wnile exercising ST ANTI?. 2 
to the maximum extent possible. 

Zacn experimental subject will go through a 

short STANTZR2 indoctrination session during which he will 
be given a summary cf the theory and implementation cf 

A3 



STAGERS, a three t briefing tc explain the srer.eric, end 
an overview of what is expected of birr. 

Turing the course of the actual experimental 
run, the evaluator nay be assisted by the VIS operators and 
test director in the mechanism of input, output, and display 
tc remove extraneous or distracting details of the game. 

4 . evaluation 

a. Tata Collection 



The 


following data should 


be collected 


for 


each 


scenario: the 


times at which 


the data 


Da se 


is 


updated, the 


situation analysis by 


STAbriP.2, 


and 


the 



situation analysis by the operator. 

These data will be augmented by the situation 
assessments made by tne test director who, having become 
familiar with the full scenario, can assess the "ground 
truth" by having knowledge cf the actions of all platforms 
throughout the scenario. The questionnaire given to 

each operator should consist of questions designed to assess 
the realism cf the scenario and the utility cf 

STAfTIF.2 as objectively as possible, 
b. Analysis 

Comparison is made between the situation 
descriptions by STAMMIR2, the evaluators, and ground truth 
to assess the general accuracy of these descriptions. The 
questionnaire results may be presented in tabular fcrr and 
the objective comments discussed in the conclusions. 



41 



c. Anticipated Results 

Anile it tray be possible to generate saturation 
points for STAKMSmJL, manifest ir. time delays w n e r e i n the 
STA! k b’FR2 display significantly lags behind the *FS scenario, 
the serial nature cf report processing would seem to 
p rec lude this possibility, providing the query function of 
STAK. M FR 2 is not extensively usea. 

t . Ccnments end Suecial Instructions 

The scenario tape should be maintained sc that a 
given ga~e may be replicated exactly wit n zero variance for 
different operators and so that the test director can 
thoroughly determine the situations as they exist (or ere 
about to exist,. Having the sames or tape vculd allow trials 
without AlS-trained operators to input the instructions 
required by a detailed script. In addition, an cbserver 
could monitor the Blue forces in one or rore cf the 
scenarios in a run without STAMMER! and make comparisons 
with the exact run without STai v 'MZR 2. Finally, if the runs 
were net available on tape, very detailed scripts describing 
all of the actions of both the Blue ard Orange forces might 
be written for each cf the scenarios to accomplish the 
objectives of this test. 
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C. STAGE -THREE EXPERIMENTS 

1 . Objectives 

The stage-three experiment has as its objectives to 
previce an operational evaluation of the military utility of 
STAN MIR2 to a decision iraker at ar. afloat task force command 
and control center in a simulated environment and to 
evaluate whether STAMPERS improves the decision maker's 
ability to make rapid and accurate decisions. As in the 
case of the stage-one experiment, this level of 
experimentation are discussed ne re but were net performed as 
part of this research. 

2 . Resources Required 
a . Technical 

The STAMPERS and WES programs ar.d associated 
computer hardware are requiren for this exrerirent. 
b . Tisplay 

An alphanumeric display system is required for 
each team c f players and the umpire team. The GZNISCO 
system, a color graphic display, may be used to present the 
situation which is generated by WES . 
c . Scenario 

An operationally realistic scenario is 
generated to test STAMPERS in a controllable tactical 
situation. The scenario should include various 
information (course, speed, sensor reports, intelligence) 
available to a Tactical Action Officer tc describe the 
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situation as it is normally presented by 'if nS , via 
visual display and status boards. The tears v. ill be allcv.ed 
tc interact with units under their command by crder i re- 
course and speed changes, enabling various sensors, and 
executing tactical orders against an opponent. The 

umpire may insert various elements of intelligence as 
desired . 

d . Personnel 

The personnel required for the experiment 
include observer tears to act as Blue and Orange 
lorces, a scenario controller, an umpire team, and a test 
director. 

e . Supper t Program 
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APPANET TTPESCFIPT feature, records all inputs end 
outputs to selected terminals. 

f. Evaluation questionnaire 

Following a study of the measures of operational 
effectiveness which may be achieved in tr.is interactive 
experiment, an evaluation questionnaire should be 
constructed to establish the level to which these reasures 
nave been satisfied. 
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^ . General Context 

a. Concept and Need 

It is possible that the value of STAi v t v 'E?.2 as a 
decision aid ray only ce assessed in an operational 
scenario. Jcr example, it ray enable a decision raker to 
better understand the threat situation which faces his 
forces and to rake rn ore accurate and timely decisions. 
These experiments would pit two opposing decision rakers 
against each ether ir. a simulated command and control 
environment and seek to measure the value of STAPhIP.2 by 
comparing the decision makers' performance operating with 
and without STAPMZR2. 



t. General Situation and Scenarios 

The operational evaluation of STAMMR2 requires 
operationally realistic scenarios consisting cf Blue and 
Orange task forces, which would be acted out with each teen 
given the flexibility cf exercising complete control ever 
their forces as long as they do not countermand their 
ordered missions. Ecr each scenario, an operational measure 
cf effectiveness (KOI) cr multiple MOEs should be selected 
and used to assess the operational utility of STANMR2. 

Because of the free-play flexibility given to 
the Blue and Orange force corranders, the war games and the 
resulting outcomes may vary significantly from trial to 
trial. Consequently, each scenario s.nculd be replicated in 
such a way as to avoid foreknowledge by a playing team. This 
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can te done by usint- c nurber of teams 'based on t he 
availability of assets), with multiple scenarios. Ry varying 
tne use and non-use of STA N ' r 'IR2 by a tear! (considering that 
a tear is expected tc play mere than one ga^e), a ccmp arisen 
of performance based on the MCIs may be made. 

Eacn team snculd go through a brief 
indoctrination session during which the following points 
will be discussed: the theory and implementation of 
ST AkkZRA , a ’threat" briefing to explain the scenario and an 
overview of the mission, and an overview of the measures of 
ef feet iveness . 

4 . Eva lua t ion 

a . Da ta Collection 

The following data should be collected fer 
each scenario: times at which the data base is 
updated, situation assessments by STAtTIED, situation 
assessments by the teams, actions taken by the decision 
maker, and data required tc determine the selected 
operational measures of effectiveness. 

These data may be augmented by the situation 
assessments made by the test director who, having become 
familiar with the full scenario, can assess the ’ground 
truth" by having knowledge of the actions cf ell platforms 
throughout the scenario. The questionnaire given tc 
each operator should consist cf questions designed to assess 
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the realism of the scenario and the utility of 

STAfM?-2 as objectively as possible, 
b . Analysis 

Data summaries rray he trade for each scenario ar.d 
tabular displays should te made of the hCIs . 

A correlation analysis should be made with the 
time sequence of actions taken by the decision maker 
based on the situation assessments made by STAPMR2 to try 
tc determine hew much the commanders utilize 
STA.* 1 MRS and what information is most useful to them t. 
comparison of the same outcomes, as reflected by the 
i v OIs, nay be made using analysis of variance to see if 

there is any significant difference due to STAmMRS. 

A comparison may be male of the game outcomes 
and the subjective appraisals of STANMIH2 to see if there 
was a relationship between how well the forces performed and 
hew much they liked STA^MR2. 

c. Anticipated Results 

It is anticipated that there cculd be a larse 
variance in the overall end- cf-game operational measures of 
effectiveness. The possibility exists that the fv ' C E s will be 
sufficiently ambiguous that any signal due tc STArTER2 may 
not be discernible. Thus, the subjective evaluations of the 
teams may provide the only discrimination in the test, with 
no quantitative assessment of the value of STAGERS 
possible . 
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The correlation analysis o? the decision, ma.-cer's 
set ions ar.d the STAMME2 assessment should, reveal the type 

of information that is rost useful to a decision maker and 
how much he may grow tc rely on S?AtVMR2. Overall, these 
experiments should yield important information acout the 
accept ab i 1 i ty of STAMMIE2 to the decision maker and its 
utility to him. 

t . Comnents and Special Instructions 

The usefulness cf STAMP2R2 tc a decision maker may 
strongly depend on the manner in which STAh'MZH2 outputs are 
displayed tc the Blue forces. This fight be especially true 
in high density situations with a backlog of unprocessed 
senscr reports. The nature cf the display dedicated tc 
STAf-'MIEZ should be considered prior to stage three 
experiments. The use of a terrinal other than the graphic 
display terminal for query and response actions allows the 
picture tc remain undisturbed until refinement of scale or 
sequential processing is requested. The display syste~ is 
an integral part of STAWEES so the evaluation should be 
based on the most flexible display available. 

A tape record of the game should be made of the 
inputs and outputs of each game so that evaluation personnel 
can reproduce a given trial and conduct a review cf any 
point to assist in "what if” types cf analysis. 
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The sceneries used should te operationally 
realistic, and the starting conditions end measures of 
effectiveness rust be selected to exercise the full range of 
STAh’MSS capabilities ar.d to yield useful information. The 
scenarios reed not be developec just for the purpose of 
exercising STAMMHR2 . Instead, tne Blue and Crange task 
forces should reflect as closely as possible, within the 
constraints of resources, the composition of forces 
currently available or projected as desired end the types of 
missions norrally undertaken by each. 

The subjects used in the experiments nay be very 
important. For purposes cf scientific credibility, the 
subjects must nave command and control experience at high 
levels, preferably Navy Captains (C-6) or Admirals. A 
potential criticisms of experiments which attempt tc 
determine the operational utility of a system is that the 
operators' c.id not reflect the potential user population. 
This would seen especially true when the evaluations depend 
heavily on subjective appraisals. There are practical 
difficulties in obtaining such test subjects. However, the 
issue is whether tnese tests can be viewea as scientific 
experiments or demonstrations without tne operational 
realism engendered by their presence. 



V. SAMPLE EXPEP.IMNT 



A . DESCRIPTION 

A sample experiment ^as select ed from tee 
preceding discussion to illustrate both the capability of 
STAMdER2 and and tne practical application of these rethods. 

The stage-two experimental methodology was selected for 
this thesis. Selection criteria included the expertise of 
the test subjects available and the availability cf the 
facilities at the Naval Postgraduate School. The test 
subjects selected were military officers with background in 
commend and control concepts gained through formal education 
and a wide range of military experience. The Gcmrand, 
Control, and Communications Laboratory contains extensive 
graphic and text capability, and the STAf-.N’EPP and WES 
software are available at the Navel Ocean Systems Center 
(NCSC), San Diego, California. 

The Warfare Environment Simulator (WES) war game 
procedure involves the construction of a computer file which 
contains identification cf all desired players, initial 
positions, courses, and speeds. Flatforms may be selected 
from a resident data base which assigns senscr and 
weapons cor.f igu ration s to the units. Following game 
initialisation, the units may be maneuvered on command from 
the player through a computer terminal, with display 
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of exact geographic location of known units available on 
selected, graphic display monitors. Interaction cf the 
units nay he orcered by the players (as in attacks), but 
sensor reports are dependent or. detection algorithms in 
V. IS. Zor example, detections by a surface search 
radar against anot her surface vessel occurs at 2? nils 
min: run separation. Ser.scr reports are given to the player 
via a status board text display and on the geographic 
display which is updated every game minute. 

This experiment was not interactive in that the 
operator did net influence the series cf events which 
were being generated by ’.vZS . It was, however, interactive 
in the sense that the operator nad tr.e capability to select 
different status board displays end change range scales 
as desirea. The fully interactive war game as suggested in 
the third level of experiment would allow operator 
influence of the event stream. The prepared scenario 
used in this experiment, wnicn included initial 
positions, initial courses and speeds, and orders to 
the various units ere included in Appendix E. 

The scenario for this experiment, senerated in 
WES , consisted of three U.S. warships conducting a 
surface surveillance of a merchant lane in the vicinity of 
the N c r t h Atlantic, vitn neutral ard \pc ten tiai ly ' hostile 
platforms in the area. The warships, assisted by patrol 
aircraft conducting similar surveillance, were tc locate, 



cl 



track, and identify ether units in the area. The 
experimental subject, embarked' on the cruiser Belknap, ned 
STAM\EP2 available to assist in that mission. The STAf- MIR2 
startup procedure is given in Anendix A. 

The scenario was chosen based on the range of situations 
available and the relative realism of a ncc-combative 
confrontation. The scenario, tactics, sensors, ard sensor 
cnaracteris t ics are purely hypothetical . The data base 
("Appendix Ej and rules ("Appendix G] are assumptions tailored 
to this scenario, but are easily translatable to those which 
a Tactical Action Officer would recognize as having 
potential appl i catiens . The briefing of the subjects 
included an explanation cf the artificialities. 

A difficulty in the actual conduct of the experiment 
existed due tc tne nature cf the reports on which STArlhEP.2 
operates. The sensor reports received by an operator of WES 
are both visual cues cn a grannies display (lines cf tearing 
or new platform symbology), and in a table of detections 
(the Electronic Warfare Status Beard, the Surface Status 
Board, or the Air Status Board). The form of the tabular 
report is formatted as tc its numerical secuence, 
identification, and location (if applicable;. The repents as 
generated by WES are net readable by STAmF’IPB . One example 
cf the disparity is that STAPLER! recognizes latitude and 
longitude in degrees and tenths rather than degrees and 
The LIS? version cf WES, LV ES , dees report in 
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minutes . 



the 



appropriate format, tut the difficulty ir using LV/ZS » such 
es complex scenario generation and lirited output 
capability, far outweigh this advantage. 

The solution to the ferret disparity lies in the 
nature of ’a IS . with duplicate platforrs, starting at exact 
locations, given precisely the same order of instructions 
isuch as course changes or sensor status), sensor reports 
are also duplicated. That is>. the detections are based cn 
raximur ranges of detection which are unchanged and 
which are combined ir. the MS detection algorithms in the 



same 


way f 


to produce 


aupl ica te 


repo r t s 


at the same 


game 


t i^e 


. This characteristic was 


expl c i ted 


by running a 


game 


to 


a suitable 


point in 


time using 


the AH? AMT 



typescript feature which copies all terminal 

interactions as they occur. This typescript shows the 
various reports received during a game, complete with the 
time cf receipt. Synthesizing game reports, a complete 
file of reports [Appendix DJ was built, in the forrat 
from which 5TAMMIR2 could read its input data. Therefore, 
STAKMR2 was net receiving direct output from the sensors, 
but via an intermediate process net part cf eitner : .v IS cr 
ST Ab MR2 . 

S T AfTER2 , using the TIATRCNIX 4C14-1 terminal for both 
text and graphic display, was, in effect, a separate entity 
running concurrently with the gane. The comparison of the 
subject's perspective of the situation end that of STAt-TIPZ 



is still valid only as long as sirultanecus report! ng is 
carefully maintained. 

The art if icial i ty of the message input could have biased 



the 


experiment 


t c 


d r 


erne rka ole 


degree but STAMPER 


2 has 


one 


feature which a 


meliorates this 


problem. STAMMER 


does 


not 


process 


the 


nex t 


sequent ia 1 


report in the file 


until 


the 


user quits 


the 


query 


mode of 


operation. That 


is , 



until the quit' corn and is given there is no 
processing of the next sequential report. Pecause the 
experimental subject is net expected to be the actual 
operator of the STAMMER hardware, the operator can be 
instructed net tc leave tne Query mede until the appropriate 
game time (which is displayed on the input/output terminal 
for a Z S ) . 

relieving a suitable period of game time where the 
subject is encouraged to exercise STAMPERS a questionnaire 
was presented to gather the impressions end appraisal of the 
experiment in general ana STAMMER in particular. 

E . RESULTS AN 2 CONCLUSIONS 

The questionnaire supplied to test subjects, with a 
summary of results, is listed below. The intent cf the 
questions was to verify the "realism” of the scenario end 
to gather a subjective evaluation of STAMMIR2. The scoring 
scale of 1 to 1 2 was arbitrarily chosen, with Id the 
most positive or most favorable response. The test 
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sutjec ts consisted cf eight Navy and Air Force officers, 
ranging in -rade from G-3 to C-5 , who are renders of 
the Command, Control and Communications curriculum at the 
Naval Postgraduate School, Nonterey, California. All have 
deer exposed to decision theory and to an overview cf the 
naval tactical environment. The Navy participants included 
three C-4s and one 0-3. All hut one of the Navy 0-4s had 
Tactical Action Officer training. 

STANDEES QUESTIONNAIRE 



1 . Prior to the scenario pres 

STANNER2 adequately explained? 


e n t a t i o n 


, was 


the 


purpose of 




12 3 4 5 


£ 


7 


c 


9 


12 


Responses 




1 


1 


'T. 


2 




Average 


S.125 












Z . Wes 

experienc 


the scenario representative 

e with VIS or L'a'ES? 


of 


you r 


P 


re vi ou s 




1 2 3 4 5 


6 


7 


g 


9 


1C 


Responses 


1 


1 




1 


Z 


y 


Average 


C ”77 R 

C • v i w 












2 • W a s 


the scenario realistic? 














1 2 3 4 5 


5 


rm 

< 


8 


9 


1 C 


Responses 


1 




1 


1 


V— 


c 



Average S.375 






w W/ 



'.‘.ere the displays consistent with the sense? reports? 



1 2 5 4 £ 5 


r- 

f 


r' 


9 


12 


Responses 


2 


2 




1 


Average 8.373 










£. '.’’as sufficient tine available to 

reports? 


unders 


tand 


the sensor 


12 3 4 3 6 


7 


c 


9 


12 


Responses 111 


2 


1 


2 


1 


Average 7.5 










6. Were the STAKMR2 graphics clear? 










1 2 3 4 5 6 


7 


£ 


S 


12 


Responses 1 1 3 2 




* 

i 






Average 4.5 










7. Was the STAMKER2 commentary understandable? 






1 2 3 4 5 6 


7 


6 


9 


1? 


Responses 1 3 


1 


r~ 

c 




1 


Average 6.575 










5. Were the ST AM KIR 2 conclusions 

conclusions? 


ccnsis 


tent 


with 


your 


1 2 3 4 5 5 


7 


£ 


9 


12 


Responses 2 


T 

o 


r* 


1 




Average 7.25 










9. Did you use the explanation trace? 










1 2 3 4 5 6 


7 


£ 


9 


12 


Responses 1 




'Z 


'Z 

v_ 


i 



Average 8.375 
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-2 . If the explar.a ti cn trace was used, w ere the assertions 
consistent with the ccnclusior s? 

1 2 2 4 5 6 7 8 9 12 

Responses 2 3 \ 2 

Average 7.375 

11. * as the presence of $?AhMR2 distract ins? 

123456789 ie 
Responses 1 3 2 1 1 

Average 4.75 



12. V'Oula an assistant who would filter information from 
STAt-MER to the rrinirurr essential level be useful? 

1 2346678912 

Responses 11 32 

Average 5.5 

13. Is S TAMM 2 P 2 useful? 



1 2 3 4 5 



7 8 9 10 



Re so 0 ns e s 
Average 7.75 



The results shew an overall trend towards an average 
response which could be considered favorable, in the 
vicinity of 8. The exceptions provide a clear contrast 
consistent with conditions noticed during the experirent. 



Question 5 had 


the 


lowest average ranAi 


rig . 


The 


difficulty encountered 


with the 


display may 


best 


be 


described by showing 


the 


cisplay at 


three selec 


ted 


ge r e 


minutes. Jigure 1 


i 5 


the last 


display v 


itheut Eh 


tearing lines, which 


0 ecu 


rs at rinute 


2. Figure 2 


is 


T O 
L .1 — 
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display at .?ame minute 14, vhicn includes intelligence data 
(t v ZRl and NIR2), surface contact reports, IV bearing lines 
and the location cf friendly units. Figure 3 is tue display 
at game minute 45 and is a compendium cf all reports 
received to that point. Close observation of the screen as 
the graphics were being drawn showed a sequential entering 



cf the 


information as it 


was 


being drawn. Sere 


measure of 


order 


w a s discernible 


if 


tnis process was 


carefully 


follow 


ed, however the 


dis 


play is unreadable 


after it is 



completed . 

A modification to the display package is required. Cr.e 
suggestion fror the test subjects was that only the last two 
occurrences of a contact be displayed, with earlier reports 
available for historical review as directed by function hey 
selection. 

A benefit of STAMMSP.c was demonstrated during the 
experiment. In every iteration of the display, intelligence 
information in the form of contact reports received from 
external sources was present on the screen as a reminder 
tc the decision maker. Eecause a suspension concerning 
this information is created ir. memory, this information was 
available at all times for review and explanation as 
desired. This information was not subject to update in this 
scenario and did not seem to contribute to the general 
clutter discussed above. 



e c 




1 STAMPERS Graphics Display 



1 i r. u t e 
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lie. 2 STAFNiRc Graphics Display, Gane Minute 14 
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066-03N 



06S-39N 



064-73N 



064-06N 



063-37N 




032-31U 



030-71U 



029-1 1U 



027-51 U 



025-91U 



STAPMIH2 Grai.nics Display 



■e Maute 4:i 
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There appeared to be a disparity between the STALER? 
conclusions and those cf the test subjects (questions £ and 
12}. The difficulty was most apparent in cases where the 
test subjects were able to cross-fix passive detection 
bearing lines to a geographic position. Rather than conclude 
that an emitting platform exists at that point of 
intersection, STAi v MEE2 would continue to make conclusions 
based only on the individual 7W reports. Here the major 
difficulty appeared to be in the construction cf STAfTIEE 
rules rather than the functioning of the rules that exist. 
A rule correlating or position-fixing multiple passive 
detections of the same emitter would appear to be a 
relatively simple matter. 

The explanation trace feature of STAf“ ; MEF.2 proved to be 
valuable as both a decision aid and as a training tcrl. 
While the users commented (question 9) that the trace 
included extraneous information, the level of understanding 
cf the scenario increased markedly due to The increased 
appreciation cf what rules were being applied and hew 
STAi v f"'EE2 applied them. 

The low average score for question 11 does not fit 
the general pattern cf the questionnaire due to the 
wording of the question . The negative response indicates 
that STAMNIR2 was not distracting tc the user. The potential 
exists for the test subjects to became engrossed i r. the 
execution of STAKMZH2 and the explanation trace to the point 
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v ner e t ne conduct cf the 5 are becomes a t i r. c r ccn sideraticn. 
This apparently was not the case here. 

The rerainder of the avera & es appear to be gene rally 
consistent with favorable response to Doth the scenario in 
general and to STAFMIE 2 in particular. 

This experiment depends heavily on subjective 
evaluation. While the scenario and operation of STAMMERS 
appear to have been convincing to the test subjects, there 
ere no clear measures of ope rational effectiveness which can 
be quantified or analyzed. Given that the computer capacity 
is available to the location of the decision raker, STAHdIR2 
would appear to be a welcome addition to the repertoire cf 
decision aids available to a commander. While this 
addresses the original objective namely the 
demonstration and evaluation of STAh 1 f / ER2, it cannot 
fully answer the question of overall utility of the system 
to a decision maker in a real military environment. 

Due to the experimental nature of STAKMZR2, operator 
action was required for the message input procedure. 
The requirement to leave the explanation trace before the 
next message can be read is purely artificial and an 
accepted penalty in order to run the program simultaneously 
with WES. In an operational system, the mechanism for the 
transfer of data between a sensor ana the decision maker 
should be understood and automatic. The addition of 
procedural steps such as function key or terminal inputs in 
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order tc receive a sensor report may tend to te 
ignored during times of stress. In addition, this automatic 
transfer must te regulated to a suitable time period between 
reports. Reports from multiple sensors at short intervals 
would rapidly overload an operator required to view every 
report . 

STAKMIR2, as an investigation into the utility of 
artificial intelligence as applied to command and control, 
has progressed beyond the concept and technology 
demonstration phase of its existence. The somewhat 
artificial nature of the message transfer procedures is 
capable of approaching a ’real world' condition in that 
machine and human readable formatted messages are in common 
use as in the P.AINFCRM reporting system. The 
production rules are similarly close tc conditions which may 
in fact exist in an operational environment, although 
an agency, such as the Office of the Chief of Naval 
Operations, operating through the Surface Warfare 
Narrower ana Training Requirements Division (OF-92), 
who could task the Fleet Training Centers (Atlantic end 
Pacific), could further refine them to reflect mere 
realistic decision conditions. 

A decision aid, even one directed toward a unio t ue 
user ie.g., the Tactical Action Officer), may need tc be 
distributed throughout a command center. The mass of data 
presented may require the decentralization of control tc the 
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poirt where the 1AC or staff watch officer function may be 
releg’ctec. to relatively discrete areas cf 
responsibility. While the requirement for a prime decision 
maker exists, this role may well be one of command by 
negation. There is a tire penalty for reevaluating 
previously screened information in a high density 
environment. Each decision maker would then require his own 
version cf STAMM EE 2 that contains a tailored rule set. 
The present implementation of STAMMZE2 already contains 
rules which may be useful and transportable to this 
practical application. 

The benefit of this program aboard a ship, cr at a mailer 
Fleet command center, for example, could be that a "super 
agency", consisting cf intelligence and warfare specialists 
could provide a realistic assessment of situations which 
right be encountered and tailor propositions to reflect 
standardized intelligence, doctrine, and tactics in a pre- 
loaded database and system that would, in real time, be an 
"oracle" which wculd provide assistance and guidance tc the 
decision maker. In case of questionable data the review 
capability of the process by which a decision was reached 



would 


assist 


the decision maker 


in logically analyzing 


his 


own 


thought 


process. The 


training benefit cf 


the 


prcpcs 


i t i c r. r 


eview cculd increas 


e the competence cf the 


TAC 



who could see hew the 'panel of experts" apprcachea the 
problem. 



c n 



In summary, this thesis has presented background 
infcrration concerning artificial intelligence er.d 

TAf-f’ERl:. Various testing methods were presented tc 
evaluate the usefulness of ST-af^iRS and an experiment was 
conducted and examined which used cr.e of these methods. 
STAPMIE2 appears to he a useful decision aid concept with 
great potential for further research and testing. 









APPENDIX A 



ST AMMEP.2 GUILE 

The following is e procedural guide to beginning 
tne STAMMER2 program for this experiment. After legging into 
the Naval Postgraduate School Command, Control, 
and Communications Laboratory UNIX system (DEC PDP 
11/70}, a TELNET link is established to the TOPS20 system 
at Naval Ocean Systems Center, San Liege, California, under 
account name NPS2. Following the prompt, the entries 
with asterisks are entered. Carriage returns (<cr>} are 
only required as indicated. 



G stammerU <cr> 

_L0AD (TZMPEAK . COM ) * 

compiled on 24-Eeb-Sl 10:29:29 
PILE CREATSL 24-Peb-Sl 10:2fc:5c 
(DISFLGB redefined) 

(IN-LANE redefined) 

(RZSULTPR INTER redefined; 

TIMPEAXCOMS 
(SENS ORANGE reset) 

< N PS2>TEtf PEAK. COM. 1 

_(SETC HOMESHIP (SETC CWNSEIP 'BELKNAP ) ) * 

(OWNS El? reset) 
i HOMES KIP reset) 

BELKNAP 

_( STAMMER ) * 

Welcome to version 2.b of the STA.MMER TSA system. 
Memory file? (Default is MEMORY.): MEM.JF <cr> 
Memory initialized. 

Rulefile? (Default is RULES . ) : RULiS . J P hcr> - 

Rules loaded 

What file would you like to take messages fror? 
(Default is SCENE. ICE/: SCENE. JF <cr> 

Are you running on a Tektronix?no 

Do you nave a Tektronix available for display? no * 
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Passive detection, heard SFS42 at bearing 25b. 29 Tine: 12 
Associated with track EIRKELEY 

REPORT: BERKELEY was sighted in the merchant lane LANZ1 
Question? Quit 
Leaving EXPLAIN 

Passive detection. Heard SPS39 at bearing 256.29 Time: 12 
Associated with track BERKELEY 

REPORT : EERXELZY was sighted in the merchant lane LAKZ1 
Question? BREAK 
(Explain broken) 
tdribble] 
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APPENDIX 3 



DATA BASS 

This is the data Base from which STAMMS HP will draw 
its first assertions. It has been specifically tailored to 
the experiment scenario ana does act reflect real- 
world conditions. 5TAMMER2 will -update its data base as 
ressa^es are received. 



i MERCHANTLAN E LANE1) 

(LOCATION LAN El ((53.33 -32.1) (t5. 9 -27.25))) 
(C'aNSHI? BELKNAP) 

(ID BELKNAP PR I END ) 

(ID-AMPLIFY BELKNAP ML-BATTLE) 

(ID 3SRKELSY FRIEND) 
i ID- AMPLIFY BERKELEY MIL-BATTLE) 

(ID KNCX FRIEND) 

(ID-AMPLIFY KNCX MIL-BATTLE) 

( ID U001 HOSTILE) 

(ID-AMPLIFY U 0 0 1 MIL-BATTLE) 

(ID S 004 FRIEND) 

(ID-AMPLIFY S004 MIL-AUXIL) 

(ID S025 FRIEND) 

(ID-AMPLIFY S005 MIL-AUXIL) 

(ID MER1 FRIEND) 

(ID-AMPLIFY MER1 MIL-AUXIL) 

(ID MER1 FRIEND) 

(ID-AMPLIFY MER1 MIL-AUXIL) 

(IB MER1 FRIEND) 

(ID-AMPLIFY MER1 MIL-AUXIL) 

(ID MER2 FRIEND) 

(ID-AMPLIFY MER2 MIL-AUXIL) 

(ID MER3 FRIEND) 
i ID-AMPLIFY MER3 MIL-AUXIL) 

(ID S 228 HOSTILE) 

(ID-AMPLIFY S20S MIL-BATTLE) 

(ID S 226 HOSTILE) 

(ID-AMPLIFY S206 MIL-EATTLS) 

(ID S 007 HOSTILE) 

(ID-AMPLIFY S007 MIL-BATTLE) 

(ID HOST4 HOSTILE) 
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^JJ-AFPLIJY 



HGST4 Yl L-rATTLZ ) 
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APPENDIX C 



RULES 



The following are the ^reduction rules upon which 
STAMMEE2 operates. As lata from the data tase or messages 
satisfy the conditions in each of the rules (labelled - 
<ITIM>), the data stream is built. When sufficient data 
exists, the rules fire end the appropriate actions are 
carried out. The plain text PRINICRM statement is the man- 
readable explanation of the assertion which has been built 
i n STAMMEE2 . 



INHERIT 

(CONDITIONS ((ALIAS -PLAT -UNKNOWN ) 

(TYPE *PLAT *TY?) 

(ID -PLAT * I D 1 ) 

( ID-AMPLI FY -PLAT *IDMP) 

(CLASS '-PLAT -CLS) 

(MEDIUM -PLAT *M2D ) } 

ACTIONS 

((TYPE -UNKNOWN -TYP) 

(ID -UNKNOWN * I D1 ) 

( ID-AMPLI PY -UNKNOWN -IDMP) 

(CLASS -UNKNOWN *CLS) 

( MEDIUM -UNKNOWN -MED)) 

CON? 1 .3 PRINECRM 

'if an unknown is identified, it inherits the properties of 

its identification.”) 

KNOWN-PLAT 

(CONDITION'S ((SIGHTING -? -S / 

(-UNLESS- (OWNSHIP -P ) ) 

( PULL Y-KNCWN *P ) 

( ID- AMP LI ?Y -P *1)) 

ACTION S 

( ( IDENTIFIED -?) ) 

CON? 1.0 FRINPORM 
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"Park as identified if known witn certainty") 

N C T -I AS T -S I G HT I N G - V ER 2 
(CONDITIONS ((SIGHTING *PLAT -Si) 

(PREDECESSOR -SI *S2) 

C-NOT- l.SAPE-AS -S 2 NIL)) 
(’•'UNLESS* ( NCT-LAST *S2)j) 
ACTIONS 

( ( NOT-LAST *S 2 ) ) 

CONE 1.0 PRINFORP 



If a sighting has a.predecess or , tnen that predecessor is 
not the last sighting.") 



NCT-JIRST-S IGHTING 

CONDITION'S ((SIGHTING *PLAT *Sl) 

(♦UNLESS* (NCT-FIRST *Sl)) 

(PREDECESSOR *S 1 *S2) 

(*NOT* (SAPE-AS *S 2 NIL))} 

ACTIONS 

( (NOT-FIRST *S1) ) 

CON F 1.0 PRINFORP 

’If an earlier sighting occurs ..record that the previous 

sighting is not the first sighting."/ 

LAST-V IEW 

(CONDITIONS ((SIGHTING *PL AT *Sl) 

(♦UNLESS* (NOT-LAST *S1))) 

ACTIONS 

((LAST-SIGHTING ♦PLAT *S 1 ) ) 

CONF .99 PRINFORP 

"If the sighting is not followed, it is the last sighting. 

(.99)") 

FIRST-V I IV 

(CONDITIONS ((SIGHTING *PLAT *Sl) 

( *UNLESS* (NOT-FIRST ♦SI))) 

ACTIONS 

((FIRST-SIGHTING *PLAT *S1)) 

CONF .99 PRINFORP 

"If the sighting is not preceded, it is the first sighting. 

( .99)") 

SIPPLY-REACHAELF 

(CONDITIONS ((CONTACT *CONT) 

(FIRST-SIGHTING *CCNT *S1) 

(SIGHTING *PLAT *S2 ) 

(ID-APPLIFY *?L AT PIL-5ATTLE ) 
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(*NCT* ( S APS -AS *CCNT -PLAT)) 

(^UNLESS* ( CWN’SHIP -PLAT)) 

(POSITION -SI -PI) 

(POSITION -S 2 *P2,‘ 

1 TOS -SI *T1) 

(TOS -S2 *12 ) 

(SWR -PI *T1 -P2 -T2 ) ) 

ACTIONS 

( (SIMPLY-WITEIN-REACH -SI -S2)) 

CCNF .96 PRINFORM 

If a contact's sighting could travel to a MIL-BATTLE's 
sighting, then they are simply reachable. (.98)’) 



RZACEA5LE 

(CONDITIONS ((CONTACT -CCNT) 

(SIGHTING -CCNT -SI) 

(SIGHTING *PLAT -S2) 

( *NCT* (SAME-AS -PLAT -CCNT) ) 
(-'UNLESS- (CWNSHIP *PLAT)) 

( S IMPLY-WI TKI N -REACH *S1 *S2) 
(-UNLESS* ( BLCCKED-FROM *S1 *S2))) 
ACTIONS 

( (’# ITEIN-RZACE *S1 *S2)) 

CONE .97 PRINFORM 



If two sightings are within reach of each ether, and are 
not blocked by patrol overflights, then are reachable from 
each other. (.97)”) 

COULD -EE-CON! BAT ANT 
(CONDITIONS ((CONTACT *C0NT) 

(FIRST-SIGHTING *CONT *S1) 

(ID-AMPLIFY *PLAT MIL-BATTLE) 

(^UNLESS* (CWNSEIP *PLAT ) ) 

(LAST-SIGHTING *?LAT *S2) 

( WITH IN-REACH *S1 *S2 ) } 

ACTIONS 

((KNOWN-COMBATANT *C ONT } ) 

CON F .15 PRINiORM 

'if a contact's position could be reached by a knevn 
combatant, then the contact might be a combatant (.15). ) 

NOT-KNCW'N-COMEATANT 
(CONDITIONS ( (CONTACT *CONT) 

(*UNLSSS* (KNOWN-COMBATANT *CCNT))) 

ACTIONS 

((TYPE *C0NT MERCHANT)) 

CONF .45 PRINFORM 
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If e contact could not be any known combatant as 
determined by rule CCULD-EZ-CCMBATAN T ) , then it -ay be a 
Ter chant ( .45 ) . ) 

FCSS-F.PT 

(CONDITIONS ((PATROL *?TL) 

(CONTACT *CONT) 

(SIGHTING *CCNT -Si) 

(SIGHTING -PLAT *S2 ) 

(ID- AMPLIFY -PLAT MIL-BATTLE) 

( -UNLESS* (GVNSHIP *PLAT ) ) 

(SOURCE -S 2 *?TL ) 

(*NOT* (SAME-AS *S1 *S2}) 

(-UNLESS- (DISSIMILAR -CCN'T '-PLAT))) 

ACTIONS 

( (POSSIBLE-REPORT -CONT *?TL)) 

CON F .55 PR INFORM 

If a patrol sisnts a MIL-BATTLE platform, and a contact is 
similar to the . pla tf o rm , then the patrol report concerns the 
contact . ( .95 ) " ) 

BLOCKER 

(CONDITIONS ((CONTACT *CCNT) 

(SIGHTING -CONT *Sl) 

(SIGHTING *PLAT -S2) 

( ID-AMPLI FY -PLAT MIL-EATTLE) 

(-NOT- (SAMS-AS -CONT -PLAT)) 

(-UNLESS- (CWNSHIP *PLAT)) 

(PATROL *PTL) 

(♦UNLESS* (POSSIBLE-REPORT -CONT *?TL ) ) 
(SIGHTING *PTL -S3) 

(NOT-LAST *S3 ) 

(SUCCESSOR *S3 *S4) 

(FOSITICN *S1 -PI) 

(POSITION -S2 -P2 ) 

(POSITION -S3 *?3) 

(POSITION *S4 — P 4 > 

( TCS -SI *T1) 

(TOS -S2 *T2 ) 

(TOS -S3 *T3) 

(TOS -S4 *T4) 

( -OR* (CROSS PATHS -PI -P2 -?3 -P4) 

(GRAZE *P1 -P2 *F3 -P4 ) ) 

(-NOT 5 *' ( WENT- EEE ORE *P1 *T1 *P2 *T2 *F3 -T3 *P4 
*T4 ) ) 

(*NOT* (* : ENT-AFTER -PI *T1 *F2 *T2 -F3 -T3 -?4 

*T4) ) ) 

ACTIONS 

( ( BLOCKED-FRCM -SI -S2)) 

CON F .9 PRINFCRM 
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If a path between two sighting has not been detected by a 
patrol, and it would have if tney were si2htin.es c: the same 
vessel, then they are different vessels. (.9)") 

CREATEDETZCT 

COME IT IONS ((SIGHTING *?LAT *SGT) 

(♦UNLESS* ( IDZNTIFI IE -PLAT) ) 

( -UNLESS- (DETECTION -PLAT)) 

(SOURCE *SG? EV ) ) 

ACTIONS 

( ( DETECTION -PLAT) ) 

CON F 1.0 PRINFORM 

"if the source of a sighting is EW , then mark it detected.") 



CREATSCONTACT 

(CONDITIONS ((SIGHTING -FLAT *SGT) 

(♦UNLESS* (IDENTIFIED -PLAT)) 

(♦UNLESS* t CONTACT ’-FLAT)) 

(SOURCE -SGT RADAR) ) 

ACTIONS 

U CONTACT -PLAT)) 

CON? 1.0 PR INFORM' 

’if radar is the source of a platform's sighting, then the 
platform is a contact.") 

CRSATEPLAT 

i CONDITIONS ((SIGHTING -PLAT *SGT) 

(♦UNLESS* (CVNSEIP -PLAT)) 

( ♦UNLESS* (PLATFORM -PLAT))) 

ACTIONS 

((PLATFORM -PLAT)) 

COME 1.0 PRINFCRM 

"Every sighting is a platform, except the ownship.") 



SMALL -CRAFT 9 

( CONDITIONS ( t CONTACT *VHO) 

(FIRST-SIGHTING -WHO *S1) 

(SOURCE *S1 RADAR) 

(RANGE -SI -Rl) 

(LESS-TEAN -Rl £) 

(STRENGTH -SI STRONG)) 

ACTIONS 

((TYPE *WEO SUE) 

(MODE *'* HO SURFACE)) 

CON F .5 PRINFCRM 

"if the range of a strong radar sighting is less than 3 am, 
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and it is the first sisntir.s, then the contact is ness illy a 
surface sub. (.5) ) ‘ 

ShAII-CRAITS 

•CONDITIONS ( ( CONTACT *X ; 

(SIGHTING *X *SIGHT) 

(NOT-EIRST -SIGHT) 

(RANGE *SI3ET' *R) 

( LZSS-TEAN *R 16) 

(GREATER-THAN *R 9) 

(STRENGTH *S IGET WEAR ) 

(SPIED -SIGHT *SPD) 

(-NOT- (GREATER-THAN *S?D 22) i) 

ACTIONS 

( (*OR* (TYPE -X PISHING) 

(TYPE -X PATROL) 

(TYPE *X SUB) ) ) 

CONE .16 PR IN FOR Y 

If tne range cf a weak sighting is between 9 and 15, an 
the speed is less than 2.2, then the contact is possibly 
sub cr a patrol or a fishing vessel. (.15)') 

SN’AIL-CRAETb 

(CONDITIONS ( (CONTACT -WHO) 

(SIGHTING -W HO -Si) 

( MOT-I IF.ST *S1) 

(SOURCE #S1 RADAR) 

(RANGE -SI GRANGE) 

(LESS-THAN GRANGE 16) 

(GREATER-THAN -RANGE S) 

(STRENGTH *S 1 WEAR) 

(SPEED *S1 PEED ) 

(GREATER-THAN *SPEED 23)) 

ACTIONS 

( ( *0?.* (TYPE -WHO SUB) 

(TYPE *VHC PATROL))) 

CON? .3 PHIN’FCRM 

’if the range of a weak radar sighting is between 9 and 16, 
and the speed is fe reater than t 23 , then the contact is 
possibly a sub or a patrol. (.2) ) 

Si v ALL-CR*ET4 

(CONDITIONS ((CONTACT -UNKNOWN) 

(SIGHTING ^UNKNOWN -S IGETING1 ; 

(LAND-DIST -SIGHT I NG1 *DIST) 

(SOURCE -SIGHTING 1 RADAR) 

(RANGE *S I GET I NG1 -RANGE) 

(LESS-THAN -RANGE 9) 

(GREATER-THAN -RANGE 3) 
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(STRENGTH *S IGHTI «G 1 WEAK) 
l LESS -THAN *DIST 52)) 

ACTIONS 

((*0?.* (TYPE -UNKNOWN SUE) 

(TYPE ^UNKNOWN S HORE-PATHOL ) 

(TYPE ^UNKNOWN PLEASURE) 

(TYPE ^UNKNOWN COMMERCIAL) 

(TYPE *UN KNOWN LANDING))) 

CONE .1 PR INF GRP 

'if the range cf a weak sighting is between 2 and 9, and the 
distance from land of the sighting is less than 52, then the 
vessel may be a sub, a patrol, a .pleasure craft, a landing 
craft, cr a commercial craft. {.!)") 

SMALL — C RA^TS 

(CONDITIONS^ (CONTACT ^UNKNOWN ) 

(SIGHTING -UNKNOWN -SIGHTING) 

(LAND-DIST -SIGHTING *DIST) 

(SOURCE ^SIGHTING RADAR > 

(RANGE ^SIGHTING GRANGE) 

(LESS-THAN GRANGE S) 

(GREATER-THAN GRANGE 3) 

(STRENGTH *S IGHTI NG WEAK) 

(GREATER-THAN *DIST 50)) 

ACTIONS 

( (TYPs"*UNKNOWN SUB)) 

CON? .35 PRINFCRM 

'if the range of a wea£ radar sighting is between 2 and 9, 
end the sighting is further.ttan 50 miles from land, then 
the contact is a sub. (.35)") 



SMALL -CRAFT 2 

(CONDITIONS ((CONTACT -UNKNOWN ) 

(SIGHTING ^UNKNOWN ^SIGHTING) 

(NOT-EIRST ^SIGHTING) 

(SOURCE ^SIGHTING RaDAR) 

(RANGE ^SIGHTING GRANGE) 

(LESS-THAN GRANGE 3) 

(STRENGTH ^SIGHTING WEAK) 

(SPEED ^SIGHTING -SPEED) 

(*NOT* (GREATER-THAN *S?ESD 3})) 

ACTIONS 

((-OR- (TYPE #UN KNOWN DEBRIS) 

(TYPE -UNKNOWN SUB) 

( TYFE ^UNKNOWN BUOY))) 

COM? .12 PRINFORM 

"if the week radar sighting is not known to be moving faster 
than 3 knots, then the contact is either a buoy, a sub, or 
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debris . ( .12 ) " ) 

Si' ALL-C HAITI 

(CONDITION'S ((CONTACT ^UNKNOWN) 

(SIGHTING *UNKNC'aN -SIGHTING) 

(NOT— i IRST ^SIGHTING) 

(SOURCE -SIGHTING RaDAR) 

(RANGE -SIGHTING GRANGE) 

(LSSS-TEAN GRANGE 3) 

(STRENGTH *S IGHTING WEAK) 

(SPEED -SIGHTING *S?EED) 

(GREATER-THaN -SPEED 3)) 

ACTIONS 

((TYPE -UNKNOWN SUB) 

(NODE ^UNKNOWN PERISCOPES CHS SNORKEL)) 

CONE .6 PR IN JOHN. 

"if the week reder sighting is moving ct greeter then 3 
knots, then tne contact is a sub in either periscope or 
snorkel mode. {.£)") 

ID-LANE 

(CONDITIONS ((SIGHTING *SHIF *S IGHTING) 

(MERCHANT LANE -LANS) 

(PLATFORM *SE I P ) 

(LOCATION *LANZ *LANELOC) 

(POSITION -SIGHTING *PCS ) 

(IN-LANE -LANELOC *POS ) ) 

ACTIONS 

( ( IMS I DZ-A-MERCEANTLANZ -S IGHTING , 

(^REPORT* -SHIP ” was sighted in the merchant lane " -LA.N 
CCN? 1.2 PRINFOPM 

"If a ship is sighted within some merchentlene , then record 
that it is inside that lane.") 

INS IDI-A.-STORM 
(CONDITIONS ((STORM *STCRN ) 

(PLATFORM -SHI?) 

(-UNLESS* i I DENT IE I ED *SEIP}) 

(LOCATION *STCRM *STMLCC) 

(SIGHTING *SHIP *S IGHTING) 

(POSITION *S IGHTING #POS } 

( INSIDE *?OS *STMLCC) ) 

ACTIONS 

((TYPE *SEIP MERCHANT) 

(^REPORT* *SH I P " was sighted inside " *STCRN)) 

CONE -.25 ?R INFORM 

"if a ship is sighted inside a storm, then the ship may not 
be a merchant. (.25) ) 
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CICS 2- POPUP 

(CONDITIONS ((CONTACT -SLIP) 

( FIRST-SIGHTING *SHIP ^SIGHTING) 

(RANGE ^SIGHTING GRANGE ) 

(LESS-TEAN -RANGE 12) ) 

ACTIONS 

((TYPE * S E I P MERCHANT)} 

CON? -.2 PR IN FORK 

‘if tiie first sighting of a ship is within 12 nn , men it 
nay not be a merchant. (.2)") 



DISTANT-POPUP 

(CONDITIONS ((CONTACT -SLIP) 

(FIRST-SIGHTING *SHI? -SIGHTING) 

(RANGE -SIGHTING GRANGE) 

(GREATER-TEAN -RANGE 30)) 

ACTIONS 

((TYPE *SHIP MERCHANT ) ) 

CONE -.2 PRINFCRM 

If the range of the first sigating is greater tnan 20 nm, 
then the ship mignt not be a merchant. (.2)") 



COURSE-CHANGED 

(CONDITIONS ((CONTACT *SHIP) 

(SIGHTING *SEIP -S IGETING1 ) 

(NOT-FIRST *SIGHTING1) 

( NOT-IAST *3 IGHTI NG 1 ) 

(SUCCESSOR *S IGET ING1 -SIGFTING2) 

(COURSE *S IGHTI NG 1 *COURSEl) 

(COURSE *S IGKTING2 *C0URSE2) 

(^UNLESS* (ROUGHLY-TEE-SAME-COURSE-AS -COURSZ1 

♦CCURSZ2)) ) 

ACTIONS 

((TYPE *SEIP MERCHANT ) ) 

CON F -.3 PRINFCRM 



If the course has changed t> s ignif icantly , then the sighting 
may not be a merchant. (.3)”) 

SPEED -CHANGED 

•CONDITIONS ( (CONTACT -SHIP) 

(SIGHTING *SEIP -SIGHTING) 

(NCT-EIRST -SIGHTING) 

( NOT-LAST ^SIGHTING) 

(SUCCESSOR -SIGHTING *SIGETING2) 

(SPEED *S IGHTI NG -SPEZD1) 

(SPEED *S IGHTI NG2 -SPEED2 ) 

(-UNLESS- (RCUGHLY-THE-SA.M E-SPEED- AS *S?ErDl 



?9 



ACTIONS 



*SPZZB2) ) ) 



( (TYPE *SEIP MERCHANT) ) 
CONE -.3 PRINFCRM 



If the speed has changed, significantly , then the sighting 
rr ay net be a merchant. (.3)") 

F A S T E R ■ -T H A M ■ - A ■ -M E R C H AN T 
(CONDITIONS ((CONTACT -SHIP) 

(SIGHTING -SHIP -SIGHTING) 

(NOT— jflRST ^SIGHTING) 

(SPEED ^SIGHTING *SPESD) 

(GREATER-THAN *SPEED 25)) 

ACTIONS 

((TYPE *SKIF MERCHANT)) 

CONE -.25 PPINFCRM 

'if the speed,, is greater than 25 knets, then it is net a 
nercha nt . ( . 25 ) " ) 



SLC'vER-THAN-A-MZRCHANT 
■CONDITIONS ((CONTACT *SEIF) 

(SIGHTING -SHIP -SIGHTING) 
(NOT-EIRST -SIGHTING) 

( SPEED ♦SIGHTING -SPEED) 
(LFSS-THAN *SPZED S)) 
ACTIONS 

((TYPE *SHIP MERCHANT)) 

CONE - .15 PR INFORM 



'if the speed M is less than 9 knots, then it may net be a 
merchant . ( . 15 ) " ) 

a?C H “PLAT 

v CONDITIONS ( ^ FIRST- SIGH TING *?LAT1 *SGT1) 

(NCT-LAST *SGT1 ) 

(SUCCESSOR *SGT1 -SGT1S ) 

{ LAST-SIGHTING -PLAT2 -SGT2 ) 

(♦UNLESS* (OWNSHIP -PLATl ) ) 

( -UNLESS* (O'A'NSHIP *PLAT2 ) ) 

(. ♦NOT** (SAME-AS -PLATl -PLAT2 ) ) 

(POSITION *SGT1S *POSlS) 

(SPEED *SGT1S *S PD 1 ) 

(POSITION *SGT1 -POSl ) 

(COURSEERCM -POSl *PCS1S *CP.Sl) 

(TOS -SGT1 *Tl) 

(POSITION *SGT2 *P0S2) 

(TOS -SGT2 *T2) 

(LESS-THAN *T2 *Tl) 

( COURSSFROM *P0S2 ♦POSl *CRS2) 
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(SPZZDERCM -PCS 2 *T2 *P0S1 *T1 *SPD2) 

( ROUGHLY-THE-S AME-CCURS Z-AS *CRS1 *CRS2) 

( ROUGRLY-TEE-S AME-S PE ZD -AS *SPD1 *S?D2)) 

ACTIONS 

( (ALIAS *?LAT2 *PLAT1) ) 

CON? .5 P? INFORM. 

"if tne course and .speed, of two sightings are roughly the 
sane, and if one sighting's position would be the ether 
sighting's extrapolated position, then the two sightings are 
of the same vessel. (.5) ) 

CUTS I DE-ALL-LANES 

(CONDITIONS ((SIGHTING -SHIP ^SIGHTING) 

( -UNLESS- ( IDENTIFIED *SHI?)) 

(PLATFORM -SHIP) 

(-UNLESS- (MEDIUM ’-SHIP AIR)) 

( -UNLESS* ( I NS IDZ-A-MERCHANTLAN Z ^SIGHTING))) 
ACTIONS 

((TYPE '-SHIP MERCHANT ) ) 

CON? -.08 PR I N FORM 

"if e sighting is outside all merchant lanes, then the 
vessel might not be a merchant. (.08)’) 

STOP 
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APPENDIX 



MESSAGES 



This is 



which STANTER2 



•ill 



the xessage file 
read, in sequential order (vice time order), during the 
execution of the scenario. It was built by putting yps 
generated reports into STAhMEP.2 readable forn. 



( BELKNAP 64.316 -28.97 0) 

(BERKELEY RADAR 64.3 -29.285 03) 

(KNOX RADAR 64.3 -28.816 23) 

(KSR1 RADAR 64.2 -30.3 0) 

(MER2 RADAR 64.832 -29.167 0) 

(1021 2W 270.0 K57 04) 

( E021 EXTERNAL 274.0 BERKELEY 54.3 -29.10 05) 

( E20 2 EW 92.0 K3 12) 

( E022 EXTERNAL 90.0 3ERKELEY 64.250 -29.15 14) 
(E001 EXTERNAL 267.0 BERKELEY 64.333 -29.133 13) 
(BELKNAP 64.4 -28.97 20) 

(KNOX RADAR 64.367 -25.734 20) 

( E20 1 EXTERNAL 263.2 BERKELEY 64.267 -29.167 22) 
( E222 EXTERNAL 92.2 BERKELEY 64.367 -29.167 20) 



(S204 EXTERNAL 64. 71c -29.333 26) 
( S 0 2 4 EXTERNAL 54.580 -29.435 35) 
( U 2 2 1 EXTERNAL 64.350 -26.333 32) 
(BELKNAP 64.435 -28.97 40) 
(BERKELEY RADAR 64.4 -29.267 40) 
(KNOX RADAR 64.433 -28.632 40) 

( S00 5 EXTERNAL 64.432 -32.2 5 40) 
(8004 EXTERNAL 64.567 -29.433 40) 



( E003 
( E0Z1 
( U 2 0 1 
( E003 
( S 00 5 
( S 224 
(S005 
( S 004 
(U001 
(U001 
( E004 
( S 206 
(2004 



EW 342.0 DONKA 41) 

EXTERNAL 238.0 EERXELSY 64.432 -29.267 40' 
EXTERNAL 64.316 -26.50 40) 

EXTERNAL 359.0 EERKELEY 64.467 -29.20 46) 
EXTERNAL 64.467 -30.016 45) 

EXTERNAL 64.532 -29.467 45) 

EXTERNAL 64.485 -29.984 50) 

EXTERNAL 64.616 -29.50 50) 

EXTERNAL 64.350 -28.646 50) 

EXTERNAL 64.350 -26.646 50) 

EXTERNAL 47.0 KNOX 64.5 -26.550 57) 
EXTERNAL 64.848 -29.333 59) 

EXTERNAL 47.0 KNOX 64.5 -26.534 59) 
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•BELKNAP 54.570 -26.97 62 ) 

(LIBKILZY RADAR 64.5 -2S.367 60) 

(KNOX RADAR 54.5 -28.534 60) 

(1003 EXTERNAL 005.0 BERKELEY 64.534 -29.383 
( S 20 5 EXTERNAL 64.534 -2S.537 65 ) 

(SZe4 EXTERNAL 64.570 -29.60 65) 

( S 00 6 EXTERNAL 64.632 -29.333 65) 

(E004 EXTERNAL 4c. 0 KNOX 64.551 -26.465 70) 

( S003 EXTERNAL 000.0 BERKELEY 54.570 -29.433 
( U 0 0 1 EXTERNAL 64.367 -26.750 75) 

(5027 EXTERNAL 64.766 -27.656 74) 

( S 00 5 EXTERNAL 64.570 -29.615 75) 

(5008 EXTERNAL 64.267 -29.649 75) 

(BELKNAP 64.532 -26.97 50) 

(BERKELEY RADAR 64.57 -29.333 60) 

(KNOX RADAR 64.586 -26.433 60) 



APPENDIX I 



SCENAR IC 



This is tne scenario which will he generated 07 the 
Warfare Environmental Simulator (WES) program, resident in 
tne TZNEX system at NCSC, San Diego, California. This file 
sets initial ship identity, position, course and speed, 
and a basic file of orders to be carried out by each side. 
The criers, which represent contingency plans, are 
executed immediately following the initialization of W ARGAf*' 
and the entry cf PLAYER wnich is the user/interactive 

program for WES. 

1 



NORTH 

YES 

YES 

SHI? 

1 . 1 * BI3KL EERY 
N00201 AaAA 322 
54-1SN 29-05 W 330 15 

SHIP 

1 .2 KNOX XNCX 

N 20202 BBEB 901 

64-18N 2S-50W 232 15 

SHIP 

1.3 BELKN BELKN 
N02023 CCCC 922 

64- 19N 28-SSW 002 15 

SHIP 

7.1 YOROS ERZS2 
Nlllll EEEE 924 

65- 28N 29-30W 245 17 

SHIP 

7.2 SKORY HASH 

N 11 1 12 FEF? 905 
65-12N 29-22W 182 22 

SHIP 
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7.2 BUHL KCTLI 
N 1 1 1 1 2 C-GGG 926 
65-25N 27-22V £17 lg 

SHI? 

7.4 VAZNY DEG PI 

Nil 114 hkHH 927 
S4-22K 22-25V/ 142 17 

SHIP 

7.5 5SGN9 CKARL 

M1115 III! 928 
54-22N 27-52V 275 25 

SHIP 

7.6 ,V IR 1 KAZBK 

N22221 H^HH 922 
64-18N 20- 18 'a' 222 15 

SHIP 

7.7 KER2 KAZBK 

N 2 22 2 2 LLLL 921 
64-52N 29-10W 215 18 

SHIP 

7.6 KER2 KAZBK 
N22223 KKKK £22 
64-18N 22-22 W 228 17 

ORDERS 

BLUE PLAN ALFA 

FOR BELKN REPORT ALL ALL TIKE 1 999 
FOR EERKL REPORT ALL ALL TIKE 1 999 
FOR KNOX REPORT ALL ALL TIKE 1 999 



PLACE A KARKER 64-12N 


22 -45 X 


TIKE 


1 


999 


PLACE A KARKER 64-2SN 


29 -20 W 


TIKE 


1 


999 


FLACE A KARKER 65-18N 


28-2 0 V 


TIKE 


1 


999 


CRANC-E PLAN ALFA 
FOR SXORY REPORT ALL 


SURFACE 


TIKE 


1 


999 



FOR VAZNY COURSE 245 TIKI 45 

FOR BURL SPEED 22 TIKE 65 

FOR VOROS PEFORT ENEKY AIR TIKE 3 94 

FOR BURL REPORT ALL ALL TINE 22 922 

FOR VAZNY REPGRT ALL SURFACE TIKE 1 999 

FCR SSGN9 REPORT ALL SURFACE TIKE 12 24 

FOR KER 1 REPORT ALL SURFACE TIKE 1 999 



BYE 
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